This is an annoying part 😕 Depending on the context, inputs.startswith(("http://", "https://")) should lead to different behavior:

• in image_to_text, a url as input must be passed as post(data=...) so that the url is loaded and sent to the inference server
• in feature_extraction, a url as input should be passed as post(payload={"inputs": ...} => a URL is a special case of string input, but still a valid one

Since _prepare_payload is agnostic of the task, it can't know in which case we are.

What do you think of modifying the method signature to

def _prepare_payload(
        inputs: Union[str, Dict[str, Any], ContentT],
        parameters: Optional[Dict[str, Any]],
        expect_binary: bool,
    )

?

For tasks that expect a binary input (image_to_*, audio_to_*), you pass _prepare_payload(..., expect_binary=True).

This way you could have a logic like this:

is_binary = isinstance(inputs, (bytes, Path))

if expect_binary and not is_binary and not isinstance(inputs, str):
    raise ValueError(...)  # should be a binary or at least a string (local path or url)

if expect_binary and not has_parameter:
    return _InferenceInputs(raw_data=inputs)

if not expect_binary and is_binary:
    raise ValueError(...)  # cannot be a binary

# else set as "inputs" in a json payload
...

hum yes you're right! actually I did not update image_to_text as we don't have any parameters or logic to either send a json payload or a raw data:

response = self.post(data=image, model=model, task="image-to-text")
output = ImageToTextOutput.parse_obj(response)
return output[0] if isinstance(output, list) else output

but of course your point is totally valid.
Having a flag seems to cover all the cases. In the beginning I though about having a Input type enum and add a input_type arg to _prepare_payload() but it's simpler to just use a expect_binary flag. I don't have a better solution either for now 😕
I will fix the suggestions and I will get back to this :)